Stochastic Games with Imperfect Monitoring

This chapter provides an introductory exposition of stochastic games with imperfect monitoring. These are stochastic games in which the players imperfectly observe the play. We discuss at length a few basic issues, and describe selected contributions. Our objective in this chapter is to provide an introductory exposition of some recent work on zero-sum stochastic games with imperfect monitoring. We will try to avoid many of the technical subtleties inherent to this type of work, by discussing at length some fundamental issues, before we proceed to introduce the basic insights of the known results. This introduction briefly recalls historical developments of the theory, discussed more extensively later, and describes the organization of the chapter. Stochastic games are played in stages. At every stage n ∈ N the players are to play one matrix game, taken from a finite set of possible games, called states. The matrix game played at stage n depends on the actions that were played at stage n− 1 and on the previous state. In the present chapter, we limit ourselves to zero-sum games, i.e., to the case where each component matrix game is a (two-player) zero-sum game. Imperfect monitoring refers to a situation where past moves of a player are imperfectly observed by his/her opponent, as opposed to perfect monitoring. Most work on stochastic games assumes perfect monitoring. ∗Laboratoire d’Analyse Géométrie et Applications, Institut Galilée, Université Paris Nord, avenue Jean-Baptiste Clément, 93430 Villetaneuse, France. e-mail: [email protected] †MEDS Department, Kellogg School of Management, Northwestern University, and the School of Mathematical Sciences, Tel Aviv University, Tel Aviv 69978, Israel. e-mail: [email protected], [email protected] ‡Département Finance et Economie, HEC, 1, rue de la Libération, 78351 Jouy-en-Josas, France. e-mail: [email protected] §We acknowledge the financial support of the Arc-en-Ciel/Keshet program for 2001/2002. The research of the second author was supported by the Israel Science Foundation (grant No. 69/01-1).